Table of contents

Introduction

Generally speaking an organization should be capable of answering it own questions better than random users, in the case of many corportaions where I worked it’s was the case to some extent.

To achieve that, you need to lay down the middle ground, you need both experts who know the answers to those questions and mechanisms to distribute their knowledge. The later range from utterly simple (ask questions, write down what you know, to the much more structured, such videos & step by step tutorials. Most importantly, corporations needs to emphasize a culture of learning,, which requires creating the psychological safety that let engineers admit to a lack of knowledge.

Sharing expertise across an organization is not a mundane task. Without a strong culture of learning & mature teams, challenges can emerge. different companies could experience a number of these challenges, especially as the company scales out, we can easily list few of them:

Lack of psychological safety

An environment in which people are afraid to take risks or make mistakes in front of others because they fear being punished for it.

Knowledge fragmentation

When different parts of an organization operate independently, without communication or shared resources, each area tends to develop its own unique processes. This siloed approach frequently results in:

• Fragmented information: each group possesses only a partial view of the overall picture.
• Duplicated effort: each group has independently created solutions for similar tasks.
• Inconsistent practices: each group has its own methods for the same activities, which may or may not be compatible.

SPOF (Single Point Of Failure)

Relying on a single person for crucial information creates a bottleneck, akin to the “bus factor.” Although this single-handed approach can appear efficient in the short term (“I’ll just do it myself”), it sacrifices long-term scalability by preventing the team from learning. This tendency also leads to a concentration of knowledge within one individual.

All-or-nothing expertise

An unbalanced team structure can emerge where you have a stark divide between those who possess extensive knowledge and complete novices, with a lack of individuals in between. This situation often perpetuates itself when experts consistently handle all tasks and don’t prioritize developing new expertise through mentoring or documentation. Consequently, knowledge and responsibilities become concentrated among the experienced members, while newcomers struggle to learn and integrate effectively.

I personally experienced this in a rapidly growing startup. Initially, with a small, trusting full-stack team, we managed everything. However, rapid scaling brought significant challenges like collaboration issues, missing documentation, and a lack of clarity. Driven by demanding clients and the need for speed, I often took on tasks myself, trying to guide others through code reviews, meetings, and answering questions – my approach to scaling knowledge horizontally. However, the constant pace of change made it difficult for everyone to keep up. My passion for continuous learning, which wasn’t universally shared, quickly led me to become a significant single point of failure across various domains like Android, Windows, backend, infrastructure, and cloud.

Parroting

Parrot-fashion adoption of patterns or code, where the form is replicated without the substance of understanding.

Tombestones

Certain parts of the code become “haunted graveyards”—places developers are wary of touching or altering because they anticipate potential negative consequences. This reluctance, fueled by fear and a degree of superstition, distinguishes these areas from the mindless copying we discussed earlier.

Eagle view

Software engineering can be defined as the multi-person development of multi-version programs.

People are at the core of software engineering : code is an important output but only a small part of building a product. Crucially, code does not emerge spontaneously out of nothing, and neither does expertise. Every expert was once a novice, and organization’s success depends on growing and investing in its people.

One-to-one advice from an expert is always invaluable. Different teams members have different areas of expertise, and so the best teammate to ask for any given question will vary. But if the expert goes on vacation or switches teams, the team can be left in the lurch. And although one person might be able to provide personalized help for one-to-many, this doesn’t scale and is limited to small numbers of many.

Documented knowledge, on the other hand, can better scale not just to the team but for the whole company. Tools such Notion enable many authors to share their expertise with a large group. But even though written documentation is more scalable than one-to-one conversations. That scalability comes with some trade-offs: it might be more generalized and less applicable to individuals learner’s situations, and it comes with the added maintenance cost required to keep information relevant and up to date over time.

Tribal knowledge exists in the gap between what individual team members know and what is documented. Experts know these things that aren’t written down. If we document that knowledge and maintain it, it is now available not only to somebody with direct one-to-one access to the expert today, but to anybody who can find and view the documentation.

In a perfect world in which the tiniest information is immediately written, we wouldn’t need to consult experts any more, don’t you think so ? Not quite, written knowledge has scaling advantages, but so does targeted human help. A human expert can synthesize their expanse of knowledge. They can asses what information is applicable to the individual’s use case, determine whether the documentation is still relevant, and know where to find it. Or if they don’t know where to find the answers, they might know who does.

The final take, tribal and written knowledge complement each other. Even perfect squads with an up to date documentation needs to communicate with one another, coordinate with other teams, and adapt their strategies over time. There is no single magical solution for all types of learning, mixing approaches is always better, only time can tell.

Setting the stage

1. Psychological safety

To learn you must first acknowledge that there are things you don’t understand. We should welcome honesty & transparency rather than punish it, but sometimes engineers are reluctant to admit they don’t understand something.

A big part of learning is being able to try things and feeling safe to fail. In a healthy environment, team members feel free asking questions, being wrong, and learning new things.

Google’s research has shown that psychological safety is the most important part of an effective team.

2. Psychological safety between teams

Asking a nearby teammate for help is easier than approaching a large group of mostly strangers. But like we’ve seen, one-to-one solutions don’t scale well. Group solutions are more scalable but also scarier. It can be intimidating for novices to form a question and ask it for a large group, knowing that their question might be archived for many years. The need for psychological safety is amplified in large groups. Each one of us has a role to play in creating and maintaining a safe environment that ensures that new comers are confident asking questions and up-and-coming experts feel empowered to help those newcomers without the fear of having their answers attacked by established experts.

According to Google, the most important way to achieve this safe and welcoming environment is for a group interactions to be cooperative, not adversarial.

3. Anti-patterns

I find Recurse Center’s social rules inspiring that’s why I’ll list few of their rules :

• No Well-actuallys: Pedantic corrections that tend to be about grandstanding rather than precision
• No Back-seat driving: Interrupting an existing discussion to offer opinions without committing to the conversation
• No subtle “-isms” It’s so easy even my grandmother could do it : Small expressions of bias that can make individuals fell unwelcome, disrespected of unsafe.
• No feigned surprise what ?! I can’t believe you don’t know what the stack is ! It’s a barrier to psychological safety and makes members of the group afraid of admitting to a lack of knowledge.

These anti-patterns can happen unintentionally.

Growing your knowledge

1. Ask questions

The more you know you don’t know “phdcomics”

if you understood one thing during this whole reading, it should be this: always be learning, always be asking questions.

One of the biggest mistakes we make is not to ask for help when we’re stuck. You might be tempted to struggle through it alone or feel fearful that your questions are too simple, I just need to try harder before I ask anyone for help. Don’t fall into this trap !

It doesn’t matter whether you’re new to a team or a senior leader: you should always be in an environment in which there’s something to learn. If not, you stagnate.

2. Understand context

Learning isn’t all about playing with new tools, it also includes developing an understanding of the decisions behind the design and implementation of existing things. Instead of saying I don’t get it and ending your thoughts there, dive deeper.

There is a famous principal called Chesterton’s fence stating that :

reforms should not be made until the reasoning behind the existing state of affairs is understood.

We engineers have a tendency to reach for the This is bad ! far more quickly than is often warranted, especially for unfamiliar source code 🤔.

Corportations are not immune to this, seek out and understand context, especially for decisions that seem unusual. After you’ve understood the context and purpose of the code, consider wether your change still makes sense. If it does, go ahead and submit an MR, if it doesn’t, document your reasoning for future readers.

Scale horizontally your questions: community knowledge

Getting one-to-one help is high bandwidth but necessarily limited in a vertical scale. And as a learner, it can be difficult to remember every detail. Do your future self a favor, when you learn something from a one-to-one discussion, write it down.

There is big chance that the future newcomers will have the same questions you had. Do them a favor, too, and share what you write down. I’ll try to discuss different forms of community-based learning, as each one have different trade-offs and sometimes complements each other.

Teams: group chats

It can sometimes be difficult to get help from the right person. Teams channels are great for this purpose, because you can ask your question to a bunch of people at once and have a quick back-and-forth conversation with whoever is available. As a bonus, other members of the group that can learn from the question and answer.

These channels tend to be devoted either to topics or to teams. Channel-driven group chats are typically open so that anyone can drop in to ask a question. They tend to have experts and can grow quite large, so questions are usually answered quickly.

Channel driven teams are smaller, open only to a few selected ones, as a result they might not have the same reach as the later but I noticed that new comers find it more welcoming and safer to use.

We saw both types and what stroke me the most was:

1. That group channels are great for quick questions, but they don’t provide much structure, which can make it difficult to extract meaningful information from a conversation in which you’re not actively involved
2. Hard to search for a given piece of data, related issues, you feel like Teams (what we used at that time) was not meant for this
3. It’s really hard to share a snippet of code, error stack, everything gets reformatted and we lose sight of what’s important
4. No closed/solved mechanism, if we go back later on to search for a similar issue and find out if a given advice/tweak was helpful, we can’t. it’s an endless stream of data

Emails

We tend to receive hundreds of emails each day, with different subjects. I’m sure we can spend days just setting up email filters to deal with the volume of notifications coming from everywhere. Some people (like me) just give up and don’t try to keep up with the flow.

Not targeting information to those who are likely to be specifically interested in it, gives birth to a high signal-to-noise ratio which can be a real problem.

Notion

Is really good for documentation, but not that great for error ticketing, code snippets, issues searching and live feedback. not to mention missing vote mechanism. I mean, you can always force the tool to do so, by scripting like we already did. But something was off, teams didn’t take it seriously, I myself, tried it so many times, but the initial cost was so high that it’s not worth it.

YAQS

YAQS (Yet Another Question System) is what we called our solution (inspired by Google’s Kubernetes F.A.Q system called discuss) making it easy for everybody to link to existing or WIP code as well as discuss confidential information.

Like Stack Overflow, YAQS shares many of the same advantages of mailing lists and adds refinements:

• answers marked as helpful are promoted in the user interface
• and users can edit questions and answers so that they remain accurate as useful as code and facts change

As a result, some mailing lists have been superseded by YAQS, whereas others evolved into more general discussion lists that are less focused on problem solving.

I must say, I was surprised how well it worked, we solved many problems and hold a growing knowledge that kept having more pieces from different domains, sometimes even non technical shreds were added.

So what you think ? give it a shot and tell me about it

💡 so much of this was inspired by books I read, so thanks to their authors and their incredible experience :

• Titus Winters (Software Engineering at Google)
• Reed Hastings & Erin Meyer (Netflix no rules rules)
• Gitlab’s commiters (Gitlab’s remote playbook)

We shall learn from their mistakes and push forward